Skip to content

listenbrainz: Add pagination, play count aggregation, and recording_mbid fix#6484

Open
arsaboo wants to merge 6 commits intobeetbox:masterfrom
arsaboo:lb2
Open

listenbrainz: Add pagination, play count aggregation, and recording_mbid fix#6484
arsaboo wants to merge 6 commits intobeetbox:masterfrom
arsaboo:lb2

Conversation

@arsaboo
Copy link
Copy Markdown
Contributor

@arsaboo arsaboo commented Mar 31, 2026

Follow-up to #6471 — fixes three remaining issues with the listenbrainz plugin:

  • Aggregate listen events into actual play counts. ListenBrainz returns individual listen events, each mapped to playcount: 1. Without aggregation, the final listenbrainz_play_count is always 1 regardless of actual listens.
  • Paginate through all listens. The API defaults to 25 results per request. Now fetches up to 1000 per page and loops via max_ts until all listens are retrieved.
  • Use recording_mbid from mbid_mapping when present. Previously mbid was left as None when the mapping existed, falling back to expensive MB API lookups unnecessarily.

Fixes #6469 (remaining issues after #6471)

  • Changelog. (Add an entry to docs/changelog.rst to the bottom of one of the lists near the top of the document.)
  • Tests. (Very much encouraged but not strictly required.)

@arsaboo arsaboo requested a review from a team as a code owner March 31, 2026 21:20
Copilot AI review requested due to automatic review settings March 31, 2026 21:20
@github-actions
Copy link
Copy Markdown

Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

grug see PR try make listenbrainz import correct and less wasteful. It add pagination so not stuck at 25 listens, it aggregate listen events into real play counts, and it use recording_mbid from API mapping to avoid extra MusicBrainz lookup.

Changes:

  • paginate ListenBrainz listens (loop pages via max_ts)
  • aggregate individual listens into per-track playcount before calling update_play_counts
  • add tests for pagination, aggregation, and recording_mbid behavior (plus changelog entry)

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 8 comments.

File Description
beetsplug/listenbrainz.py add pagination, aggregation helper, and recording_mbid fix for play count import
test/plugins/test_listenbrainz.py add tests for new pagination/aggregation and mapping behavior
docs/changelog.rst document ListenBrainz bugfix behavior change

Comment on lines +127 to +133
if max_ts is not None:
params["max_ts"] = max_ts
if min_ts is not None:
params["min_ts"] = min_ts

response = self._make_request(url, params)
if response is None:
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grug see doc say min_ts and max_ts no mix. but code will send both if caller pass both. grug want guard: raise ValueError (or ui.UserError) when both set, so no weird API result.

Copilot uses AI. Check for mistakes.
Comment on lines +123 to +129
all_listens = []

while True:
params = {"count": per_page}
if max_ts is not None:
params["max_ts"] = max_ts
if min_ts is not None:
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grug see count become per_page, but no clamp to API max 1000 and no check for <=0. grug want clamp to 1000 and reject non-positive so request not fail or loop strange.

Copilot uses AI. Check for mistakes.
found_total += found
unknown_total += unknown
"""Obtain play counts from ListenBrainz."""
listens = self.get_listens()
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grug see when request fail, get_listens break and return empty list. then _lbupdate log "No listens found" like user have none, but really API error. grug want error path distinct: log warning/error and bail, or return None/raise so caller can tell.

Suggested change
listens = self.get_listens()
listens = self.get_listens()
if listens is None:
# grug: error path different from "no listens" path
log.error("Failed to fetch listens from ListenBrainz; aborting import.")
return

Copilot uses AI. Check for mistakes.
Comment on lines 147 to 153
# Use the oldest listen's timestamp for the next page
max_ts = listens[-1]["listened_at"]

return all_listens

def get_tracks_from_listens(self, listens) -> list[Track]:
"""Returns a list of tracks from a list of listens."""
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grug worry pagination no progress guard. max_ts set to oldest listened_at, but if API treat max_ts inclusive or many listens share same second, next page can repeat and maybe loop forever / overcount. grug want make next max_ts strictly older (ex minus 1) or detect max_ts not decreasing and break.

Copilot uses AI. Check for mistakes.
Comment on lines +142 to +146

# If we got fewer than requested, we've reached the end
if len(listens) < per_page:
break

Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grug see all_listens grow big then later convert to tracks then aggregate. for big user, memory boom. grug want stream pages: aggregate Counter as pages come, no keep every listen dict in list.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

memory boom 🤣

Comment on lines +73 to +81
play_counts: Counter[str] = Counter()
track_info: dict[str, Track] = {}
for t in tracks:
mbid = t.get("mbid") or ""
artist = t["artist"]
name = t["name"]
album = t.get("album") or ""

key = mbid if mbid else f"{artist}||{name}||{album}"
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grug see aggregation key build with string join and '||'. if artist/title/album contain '||' then collision and wrong counts. grug want use tuple key (mbid) or (artist,name,album) instead of string concat.

Suggested change
play_counts: Counter[str] = Counter()
track_info: dict[str, Track] = {}
for t in tracks:
mbid = t.get("mbid") or ""
artist = t["artist"]
name = t["name"]
album = t.get("album") or ""
key = mbid if mbid else f"{artist}||{name}||{album}"
play_counts: Counter[str | tuple[str, str, str]] = Counter()
track_info: dict[str | tuple[str, str, str], Track] = {}
for t in tracks:
mbid = t.get("mbid") or ""
artist = t["artist"]
name = t["name"]
album = t.get("album") or ""
if mbid:
key: str | tuple[str, str, str] = mbid
else:
key = (artist, name, album)

Copilot uses AI. Check for mistakes.
Comment on lines +73 to +76
},
{
"mbid": "m2",
"artist": "B",
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grug see new test lines very long (nested dict in one line). repo Ruff check E501 unless file ignored, and this file not ignored. grug want wrap these dict literals across lines so lint pass.

Copilot uses AI. Check for mistakes.
Comment on lines +77 to +80
"name": "T",
"album": "Bl",
"playcount": 1,
},
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grug see same long-line problem for page2 listen dict too. please wrap line to stay under configured max line length so Ruff not yell.

Copilot uses AI. Check for mistakes.
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 31, 2026

Codecov Report

❌ Patch coverage is 65.51724% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.10%. Comparing base (712bada) to head (fafaf73).
⚠️ Report is 27 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
beetsplug/listenbrainz.py 65.51% 16 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6484      +/-   ##
==========================================
+ Coverage   70.01%   70.10%   +0.08%     
==========================================
  Files         146      146              
  Lines       18505    18545      +40     
  Branches     3010     3020      +10     
==========================================
+ Hits        12957    13001      +44     
+ Misses       4920     4911       -9     
- Partials      628      633       +5     
Files with missing lines Coverage Δ
beetsplug/listenbrainz.py 52.66% <65.51%> (+17.77%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@snejus
Copy link
Copy Markdown
Member

snejus commented Apr 3, 2026

@arsaboo could you resolve the Copilot comments that you've addressed?

@JOJ0 JOJ0 added the plugin Pull requests that are plugins related label Apr 4, 2026
@snejus snejus added listenbrainz listenbrainz plugin and removed plugin Pull requests that are plugins related labels Apr 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

listenbrainz listenbrainz plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

listenbrainz: the plugin appears to be broken

4 participants